1. Technical Field
This invention relates to the field of data indexing and retrieval, and in particular to systems and methods for managing the data stored on computer networks and the indices used to track this data.
2. Background Art
Computer networks are capable of storing ever-increasing amounts of data. In order to manage this data effectively, these networks employ various indexing schemes to represent the stored data in a manner that facilitates information searching and retrieval. Currently available management schemes rely on a central computer to coordinate the distribution of indices and data among the computers of the network. The data and indices are typically distributed among a network of server computers, which can be accessed from other computers, e.g. personal computers, on the network.
One shortcoming of these conventional, centrally coordinated management schemes is that they do not scale well. That is, they cannot be expanded to accommodate the ever-increasing amounts of data available on computer networks without substantial modifications to the way the data and indices are handled. The overhead borne by the central computer for tracking and coordinating the distributed information impedes the efficiency with which these systems operate. This problem appears in the intranets used by organizations to store their data, and it is exacerbated where data from the much larger internet is stored and tracked by these intranets.
Another shortcoming of these distributed systems is that only relatively powerful server computers are used to store indices and data. Servers are used because they can store more information than personal computers, and this minimizes the number of computers that the managing computer must coordinate. However, confining the distributed information to servers ignores the substantial processing and storage power available on the many smaller computers that form the majority of nodes of a network.
Agents are available to scour the internet for data specified by a user. However, these agents merely locate and retrieve the specified data from network sites. They are not designed to manage, index, or distribute data and index files.